This invention relates to visual displays which preferably can combine generated images with a view of the environment surrounding a user and transmit such combined visual information to the eye position of the user.
It is often desirable to provide people with visual information. Frequently, one wishes to superimpose such visual information upon a view of the real world. In other applications, it is desired to shield the user from a view of the environment, providing an exclusive view of the visual information. Such displays may include a number of components including, in a form known as a folded catadioptric display, an image generator, a beam splitter which receives the image light from the image generator, often via a series of lenses and other optic elements and sends a fraction, designated the reflected fraction, of such image light to a reflective combiner that is either non-transmissive or both allows light from the real world to pass through such combiner and reflects the image light such that both the real-world light and the image light are transmitted to the eye of the user through the beam splitter, often via another series of lenses or other optical elements. The beam splitter will transmit a fraction, designated the transmitted fraction, of the image light reflected from the collimator-combiner. In embodiments in which the combiner is at least partially transmissive, a fraction of the real-world light is also transmitted by the beam splitter.
Previous devices included a number of additional components. Some devices have included corrective optical elements. Other devices have included a depixelator, e.g., as described in PCT/US94/01390 filed Feb. 7, 1994 for xe2x80x9cDepixelated Visual Displayxe2x80x9d (incorporated herein by reference). Still other devices have included apparatus for intensifying the visual display such as those described in PCT/US94/01391, filed Feb. 7, 1994 for xe2x80x9cIntensified Visual Displayxe2x80x9d (incorporated herein by reference).
In designing a system for providing a generated image to the eye or eyes of a user, various factors are often in opposition. Although it is desirable to provide an image which has high quality so as to not only provide a pleasing and attractive display for the user but also to reduce eyestrain, very often the techniques used to produce such a high quality image have been contrary to goals of an apparatus which is lightweight, low-cost, and relatively easy to design, fabricate and/or repair. Many previous devices have required expensive and heavy series of optical elements such as lenses to achieve a desired picture quality. Other devices have simply accepted a lower-quality image in order to achieve goals of low-cost or light weight. For example, some previous devices have failed to provide a flat focal field to the user. Others have resulted in a poor image contrast, particularly at high viewing angles of the image. Many devices have deprived the user of the full image in order to mask-off unwanted light near the edge of an image generator.
Accordingly, it would be useful to provide a personal, visual display apparatus which provides a high quality of image, which is low cost, lightweight, comfortable and has increased ease of design, fabrication, repair and the like.
In many uses of a head mounted display, it is desired to collect information regarding the position, posture, orientation, attitude, location and/or movement of the user""s head. This information can be used to control the image generated to the eyes of the user for a number of purposes, such as to provide a xe2x80x9cvirtual realityxe2x80x9d or simulated environment to the user, to allow the user to navigate through various displayed images to observe a simulated object from various angles and the like. For example, this information can be used to control the characteristics of the image shown in a head-mounted display or other virtual reality device, such as to produce changes in the image that would correspond to movement of the user""s head. For example, in a flight simulator program, when the user""s head rotates from a straight ahead position to a 90xc2x0 left position, the display should change from a display simulating a forward view, out the cockpit window, to a display simulating a view over the left wing of the aircraft.
Head trackers have been attempted in the past, such as those described in U.S. Pat. Nos. 5,373,857; and 5,345,944. However, many previous head trackers have suffered from a number of deficiencies. In some devices, head trackers were formed as an integral part of a head-mounted display and were not readily detachable. This, however, led to head-mounted displays which are heavier than necessary when used in applications where head tracking is not needed or desired. Furthermore, integral head trackers add to the cost of a head-mounted display in a manner which may be unnecessary for some uses when head tracking is not desired.
Some head tracking devices have included mounting some of the components in positions which do not provide a desirable balancing of weight in the head-mounted display device, such that there may be an undesirable neck strain for the user.
Some previous head trackers provided communication to the computer which was difficult for the end user to install and/or awkward in operation. For example, in some devices, it was required that the user install a card inside the chassis of a personal computer (PC) which, effectively, meant that many users would need to make a trip to a computer service facility.
In a configuration in which communication from the head tracker to the computer is via a cable which is separate from the head-mounted display cable, the user must wear a device which is xe2x80x9ctetheredxe2x80x9d via two separate cables, which can tangle and inhibit free movement of the user. In devices where a single cable provides for communication of both head tracker and video information, the head tracker was not readily detachable from the head-mounted display.
In some previous devices, the output from the head tracker was provided in only a single format and the software which employed head tracker information had to be written so as to accommodate that data format.
In some devices, a head tracker used magnetic sensors. While magnetic sensors can be useful in many situations, head tracking information may be distorted by spatial variations and/or changes in the local magnetic field, such as may arise from adjustment of earphones or other audio output devices.
Accordingly, it would be useful to provide a head tracker which is of modular or detachable construction, can be located in a well-balanced position, provides for communication with the computer in a fashion which is easy for the end user to install and avoids awkwardness of use, reduces or eliminates errors from magnetic field changes and/or which provides output in a number of different output formats.
One embodiment of the invention is a head-mounted display (HMD) that can project an image from an image generator such as a cathode ray tube (CRT) or a liquid crystal display (LCD) to one or both of the eyes of the observer or both transmit such an image and combine it with a direct view of the surrounding environment. The combiner images a CRT or LCD display surface mounted above the eye with a simple metallic, dielectric or holographic fold mirror reflecting the image towards the combiner.
According to one aspect of the present invention, the device simplifies the imaging optics by reducing the total number of elements. One embodiment of the invention consists essentially of an image generator such as a CRT or LCD, a combiner such as a spherical metallic, dielectric or holographic combiner or collimator-combiner and a fold mirror such as a metallic, dielectric or holographic fold mirror.
In one embodiment, few optical elements are needed to achieve the desired high quality image. Preferably, the visual display is simplified and made lightweight and less expensive by eliminating the need for certain optic elements such as relay lenses or additional corrective refractive optics. In one embodiment, a visual display provides for monochromatic and/or three-color display without refractive optics, such as by using reflective imaging optics only.
Because the weight of the device required for image display is low, it is possible to include any or all of a number of additional functions while maintaining a relatively light device, including functions such as headtracking, eyetracking, three-dimensional display decoding or other 3-D capability, ability to be used by a wide range of sizes and shapes of users, comparability with both computer and video source material, compatability with eyeglasses and the like.
In one embodiment of the invention, the apparatus provides a high-quality image while using a lightweight and visually attractive configuration. In one embodiment, the apparatus is a glasses-like structure with optics configured to deliver an image to the user""s eye position, preferably to deliver generated images to the left and right eyes of a user, and with temple pieces projecting backward on each side of the user""s head in the fashion of eyeglasses. A strap may be used to assist in holding the device in a desired location. Headphone-like loud speakers are positionable near the user""s ears to provide simultaneous video and audio to the user. In one embodiment, a brace is used to contact the user""s forehead to further assist in proper positioning and weight-bearing. This configuration is particularly useful when it is desired to leave enough space between the apparatus and the user""s eyes to accommodate ordinary eyeglasses. Preferably, various controls are available such as being mounted on the headmountable apparatus. These can include, for example, a mute button such as an audio and video mute, volume control, image selector and the like. Preferably, the temple pieces can be folded, similarly to folding of typical eyeglasses to achieve a compact storage or carrying configuration.
In one embodiment, the display device is modifiable by addition, preferably snap-on addition, of further components. For example, a light shield may be snapped on to convert a see-through display into an immersion display. Head position tracker components can be added to provide an indication, e.g., to a computer and/or video game device, of the orientation, movement, and/or location of the user""s head.
In one embodiment, an optical element is configured to reduce or eliminate loss of image contrast such as that which may occur from an image generated by a backlit LCD display, particularly near the edges of the image. In one embodiment, a color display with high density is provided by the combination of a white or monochrome display coupled with a color shutter. In one embodiment, the shutter is located spaced from the screen or output plane of the image generator, and may be positioned substantially immediately in front of the eyes of the user. In another embodiment the shutter is located adjacent to the image generator output plane or to an optical element, such as a field curvature correction lens, which is adjacent to the image generator output plane.
According to one embodiment, shrouds provide proper shielding, holding and alignment of various components of the device. A shroud can be configured to not only block stray light but also to hold, for example, a fold mirror and/or combiner. In one embodiment, two or more masks are provided for the image generator to achieve elimination of unwanted light source angles without masking desired portions of the image. Preferably, such masks are provided as a unitary piece which can also function to hold other elements such as the LCD or other image generator, a light source, a lens, etc. By providing for an adjustment in the position of such a holder, it is possible to correct positional disparity, particularly vertical positional disparity between the left and right image generators when the apparatus is used in a binocular fashion.
According to one embodiment of the invention, tracking devices are configured so as to permit the user to attach and detach the tracking devices as desired. In one embodiment the head tracker can be mechanically coupled to a head-mounted display by a simple coupling such as a hook and loop type of coupling. Preferably, the head tracker can be located at the back of the head to act as a partial counterweight to the video electronics and optics which are typically mounted near the front of the head. In one embodiment the head tracker contains pass-through wiring or circuitry so that a single cable provides communication between the host computer and the tracker/HMD but such that, when the tracker is detached, a single cable can be unplugged from the tracker and coupled to the computer for providing communication between the computer and the HMD. Preferably communication with the computer is accomplished via commonly-available ports such as a serial port, video port, and/or audio ports, and it is not necessary to install a card or other hardware device in the host computer.
Preferably, data is output from the tracker in a variety of user-selectable formats including hardware- filtered, hardware- and software-filtered, Euler, and mouse-emulation formats, in either polled or streaming fashion.
A number of sensor technologies can be used for sensing head movement or position including magnetic sensors, inertial sensors and mechanical sensors. In one embodiment, inertial sensors are used for both yaw sensing and pitch/roll sensing such as by using a rate gyro for yaw sensing and two accelerometers for sensing pitch and/or roll, respectively. Other embodiments include using a rate gyro for detecting yaw and one or more gravimetric sensors for detecting pitch and/or roll, and using a magnetic sensor for detecting yaw and inertial sensors (such as one or more accelerometers) for detecting pitch and/or roll. In another embodiment, magneto resistors are used for yaw sensing and gravimetric sensors are used for pitch and roll sensing. In one particular embodiment, three pairs of orthogonally-mounted magnetoresistive sensors, each pair forming two legs of one of three Wheatstone bridges provide yaw detection. Preferably, fewer than three normalizing coils are required, such as by using one normalizing coil for normalizing two of the three pairs of magnetoresistors.